Many computer program bugs are found very late in the product cycle, or in a worse case, by customers. Indeed, given a large enough market, software consumers will find numerous bugs in software programs, which can reflect poorly on the developer. Thus, an important part of software development is testing to eliminate bugs in software, and to have the software designed to otherwise handle unusual circumstances. That is one reason why Beta testing is used, so that by sheer numbers, many users (who understand that bugs are likely early in the development process) can help to debug a product before it is sold to consumers.
Beta testing is only one way software is tested, and can only be done when the product under development is reasonably stable and safe enough to give to those who will use it in the real world. To get to this point, and also to find bugs that even large numbers of Beta testers may not find, software producers also run their programs through pre-arranged tests.
While these tests find many bugs, there is no way for developers to realistically anticipate each of the possible combinations of actions that can cause a bug. Testing all combinations is not possible. By way of a general example, consider an application program's or shell program's I/O (input/output) requests directed towards a file system or remote server; it is virtually impossible to test all I/O combinations since they are essentially infinite, especially if timing among the I/O requests is a factor.
As a more particular example, consider a filter driver component such as a quota filter driver that limits the amount of space a user may consume on a shared storage volume. The test team that tests the quota filter driver is not responsible for testing the operating system shell or a suite of application programs such as Microsoft® Office. Likewise the shell and program suite teams are not responsible to test quota filter drivers. Addition of another filter driver, such as an antivirus product, can also change the way I/Os behave. Essentially, there are no tests as to whether the I/Os triggered by application programs and/or the shell work properly, let alone when using a certain filter driver or combination of filter drivers, other than tests performed manually, which are very time consuming and error prone. Further, it is not practical to redo these manual tests on a regular basis, even though various versions of programs and filter drivers are regularly released.
In sum, the I/Os generated by an application program can uncover program bugs in the program as well as in lower-level filter drivers that handle corresponding I/Os requests below the program. However finding and fixing those bugs often requires significant manual testing to recreate the actions that first found the bug. What is needed is a way to automate testing tasks that otherwise have to be done manually when testing the compatibility of kernel-mode components with user-mode programs.